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ABSTRACT 

The multimodal correlation between different diagnostic exams, the intraoperative calibration of pointing 
tools and the correlation of patient’s virtual models with the patient himself are some examples, taken 
from the biomedical field, of a unique problem: determine the relationship linking representation’ of the 
same object in different reference frames. Several methods have been developed in order to determine this 
relationship, among them, the surface matching method is the one that gives the patient minimum 
discomfort and the errors occurring are compatible with the required precision. The surface matching 
method has been successfully applied to the multimodal correlation of diagnostic exams such as CT, MR, 

PET and SPECT. Algorithms for automatic segmentation of diagnostic images have been developed to 
extract the reference surfaces from the diagnostic exams, whereas the surface of the patient skull has been 
momtored, in our approach, by means of a laser sensor mounted on the end effector of an industrial robot. 

An integrated system for virtual planning and real time execution of surgical operation, that is described in 
this paper, has been realized at the Department of Mechanical Innovation and Management of the 
University of Padova, Italy, in cooperation with the Neurosurgical Division of the Hospital of Treviso, 

Italy. 


1. INTRODUCTION 

In recent years, systems for surgical planning and 
intraoperative assistance have been widely used. In 
these systems a CT or MR exam is used to reconstruct 
a virtual model of the anatomies of interest. Dedicated 
software programs allow the surgeon to simulate on the 
virtual model the approach strategies, to determine and 
study the target points, the trajectories, etc. Several of 
these systems allow the correlation among different 
diagnostic exams. To correlate different diagnostic 
exams it is necessary to compute the transformation 
matrix between the reference frames associated to each 
diagnostic modality. A classical example of it in the 
neurosurgical field is the stereotactic head frame. 
Several types of localizers, mounted on the base ring of 
the head frame, fixed to the patient's skull, produce 
artifacts in the diagnostic images. Localization of the 
artifacts in the images allow to calculate the coordinates 
of each voxel of the scanned volume with respect to the 
head frame. In this way a correlation can be found, 
between a diagnostic image and the stereotactic system. 
Using different types of localizers, this correlation can 
be established between each diagnostic modality and the 
stereotactic reference frame, and consequently among 
the diverse diagnostic exams. In more recent years, 
newer correlation methods based on artifacts produced 
by fiducial markers fixed to the patient's skin or 
implanted in the bone were developed. The data gotten 
from the planning stage can be used during surgery only 
if a registration method is found, correlating the 
reference frame of the "real" patient with one of the 


diagnostic exams. Such a registration is implicitly 
achieved if a stereotactic head frame is used, while if 
fiducial markers are employed it is necessary to use a 
three-dimensional digitizer to determine the position of 
the fiducial markers in the real world. A correlation 
matrix can be computed by surface matching only if 
equivalent reference surfaces can be extracted from 
different diagnostic exams. Among all the surfaces, the 
CT is taken as a reference, because it is characterized 
by veiy low distortion. The same procedure can be 
applied to intraoperative matching, determining, by 
means of suitable sensors, the real surface to be 
correlated to the CT. In this paper methods for image 
acquisition and processing for multimodal correlation 
and intraoperative matching by means of surface 
matching techniques are described. These methods are 
general enough to be applied to the calibration and 
adjusting of robots in other fields than the 
neurosurgery. 


2. MULTIMODAL IMAGING 

The features of each diagnostic exam are due to 
different physical quantities that are sampled. For 
instance, the CT provides a great deal of information on 
the bone tissues, but its spectrum is very narrow for soft 
tissues (about 100 Hounsfield numbers out of 2001). 
On the other side, MR images show soft tissues very 
efficiently, but bones are not visible. Blood vessels are 
usually detected using Digital Angiographs. Even the 
most expert radiologist would find hard to mentally 
correlate the information gotten from all these 
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most expert radiologist would find hard to mentally 
correlate the information gotten from all these 
diagnostic exams. For this reason our research was 
aimed at finding a non invasive method, which could 
guarantee high reconstruction precision, easy to be 
implemented and general enough to be applied to 
different anatomical districts and different diagnostic 
modalities. Our work started off using fiducial markers 
fixed to the patient's skin and visible in all diagnostic 
exams; in this case the correlation between the different 
modalities was calculated on the basis of the position of 
the markers in the various reference frames. Algorithms 
for 3D-3D and 2D-3D correlation were developed and 
tested. Then, the surface matching was introduced as a 
method to minimise the transformation errors. Such a 
hybrid approach (surface matching + fiducial markers) 
revealed itself adequate for diagnostic purposes and 
can be effectively used also for intraoperative matching. 


3. INTRAOPERATIVE 
ASSISTED EXECUTION 


MATCHING AND 


The virtual model of the patient is used by the surgeon 
to define the targets, the approach directions and more 
generally the surgical tasks to execute thru the 
robotized system. The system is made of an ASEA 
BRB2000 industrial robot on which a laser system for 
distance measurement (by Microepsilon) has been 
mounted, a robotic simulation software (Robcad, by 
Tecnomatix), and a three-dimensional digitizer 
(Surgicom by Faro Tech.). The digitizer is a six d o f. 
passive arm connected to a UNIX workstation thru' a 
serial interface box. The operating room is thoroughly 
modeled in the robotic simulation environment. The 
environment is totally structured except the volume 
occupied by the patient and by the mobile medical 
devices associated to him. To guarantee the patient's 
safety, once he has been fastened to the operating bed 
thru a Mayfield (Ohio Instr.) head holder, the position 
of the fiducial markers on the patient's skull is 
determined by means of the three-dimensional digitizer. 



Figure 1- Registration of the fiducial markers. 


The coordinates of the fiducial markers are transformed 
from the digitizer's reference frame into the robot's 
reference frame by means of a transformation matrix 
which was computed during the cell definition 
procedure. In fact, the cell has been defined a priori 
localizing all object inside it, including the digitizer, 
with respect to the robot's reference frame. This allows 
to define a bounding security volume that may never be 
entered by the robot and the instruments associated to 
it. Moreover, in order to avoid interference between the 
robot and the devices inside the operating cell, all 
movements the robot is to make for the registration 
procedure are simulated before execution. The data of 
the patient's reference surface are gotten by means of a 
laser sensor installed on the robot's end effector. This 
acquisition procedure is firstly simulated utilizing the 
virtual surface extracted from the diagnostic images as a 
reference, so that the laser beam is kept perpendicular 
to the surface itself. Then, during the measurement 
procedure in the real world, the direction of the laser 
beam is adjusted on the basis of the previous 
measurements. Therefore, the measurement procedure 
in the real world does not correspond exactly to the 
simulated one, but there are minor differences in the 
robot's end effector positioning.During the simulation 
stage, the direction of the first approach to the patient, 
to begin the measurement scanning, is defined’ 
Knowing then the position of the fiducial markers, 
computed using the digitizer, an approximated 
transformation between the patient's and the robot's 
reference frame can be found. This function is used to 
limit the number of iterations of the surface matching 
algorithm, avoid the problem of local minima and 
overcome the ambiguities due to the symmetry of the 
skull. 



Figure 2 - The operating cell model with the 3D- 
digitizer, the robot and the CT surface during the 
registration procedure. 

The surface matching algorithm computes the patient- 
robot transformation minimizing the distance between 
the reference surface extracted from the diagnostic 
images and the real surface measured by means of the 
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laser sensor. This matrix is then used to transform into 
the robot's reference frame al the geometric entities 
defined in the planning stage with respect to the CT 
reference frame. In this way the real operating cell is 
completely described in the simulation environment, 
where the patient, the target points, the trajectories, etc. 
are defined. Thus, the robot's movement to execute the 
planned tasks can be simulated and the simulation 
sequence is shown to the surgeon on a monitor located 
in the operating room; if the sequence is correct, the 
execution stage can be started. The transformation 
matrix is exact as long as the patient is not moved by 
any surgical procedure executed on him. Should this 
happen, it is necessary to recalibrate the system using 
three reference points located on the head holder, which 
is assumed to be fixed with respect to the patient's head. 
In this case too, the approximate position of the 
calibration points is measured using the three- 
dimensional digitizer so that the laser sensor can be 
positioned near them. 


4. PROCEDURE FOR MEASUREMENT OF THE 
REAL SURFACE 

In this work a technique for measuring the real surface 
of the patient's skull for intraoperative matching has 
been developed, which utilizes a laser sensor and a 6 
d.o.f industrial robot. The laser sensor uses a 
triangularization method allowing to measure its 
distance from a surface characterized by a diffuse 
reflection. The precision of the measurement is 
independent from the type of material constituting the 
surface. Furthermore, the system is totally non invasive 
for the patient. 



Figure 3 - The laser sensor mounted on the robot’s end 
effector. 


The sensor is characterized by a midrange distance 
about which there is a displacement range where the 
output signal vs. distance characteristic can be 
considered linear.The sensor is a Micro-Epsilon opto 
NCDT series 1605-100 whose main characteristics are - 
Midrange 220 mm 

Displacement range ± 50 mm 


Non-linearity 300 mm 

The sensor guarantees accurate readings if the 
measured surface presents a ± 1 5 °angle with an axis of 
the sensor and ±30 with an axis perpendicular to the 
former. This limit makes necessaiy to orient the sensor 
during the measurement of curve surfaces to guarantee 
as much as possible that the laser beam be perpendicular 
to the surface in the measurement point. Since the skull's 
surface does not have a defined shape, it is necessary to 
introduce a measurement procedure that adjusts the 
direction of the laser beam on the basis of the previous 
readings. Thus, the sensor has been mounted on the 
robot's end effector: in this way the sensor can be 
positioned and oriented in whatsoever direction in the 
whereabouts of the skull. This guarantees a repeatability 
of ±0.1 mm and a precision of ±0.1 mm. As it has 
been described above, the measurement procedure 
starts with a coarse calibration of the robot with respect 
to the skull in order to define the workspace. When the 
patient has been positioned on the operating bed, the 
approximate position of the skull is determined by 
reading the fiducial markers with the three-dimensional 
digitizer. This provides an approximate transformation 
matrix that allows to begin correctly and safely the 
measurement procedure. A midrange surface is 
generated in the virtual model of the skull; this surface 
has an average offset equal to the sensor's midrange. 
The midrange surface must be external to the bounding 
security volume, that is the volume that may not be 
entered by the robot. Then, the robot positions itself in 
that point of the midrange surface which corresponds 
approximately to the top of the skull, with a direction 
perpendicular to the modeled surface. Thus, the first 
measurements with a fair precision can be made, 
allowing a self calibration of the system, that will be 
now described. Three readings are made on the vertices 
of an equilateral triangle, described on the midrange 
surface in the whereabouts of the point that has been 
determined on the top of the skull. These three readings 
allow to calculate the normal to the real surface in the 
center of the same triangle. The measurement sequence 
proceeds with a series of points located along a spiral 
on the surface. At each further step the reading 
direction is determined on the basis of the last three 
measured points. These three points are chosen in such 
a way to guarantee a good approximation in the 
determination of the local normal. However, the reading 
procedure is not critical because the skull's surface is 
regular enough. The result of the measurement is a set 
of points describing the patient's head surface in the 
robot's reference system. 


5. CONCLUSION 

Surface matching is a suitable non-invasive registration 
method for multimodal correlation and intraoperative 
matching. It guarantee, with the minimum discomfort 
for the patient, registration errors compatible with the 
required precision. It can be successful applied to 
multimodal correlation of diagnostic exams in which 
reference surfaces can be extracted thru interactive 
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segmentation programs. The same surfaces can be used 
for the intraoperative matching by means of a laser 
sensor mounted on the end effector of an industrial 
robot for the “real surface” detection. 
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